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SYSTEM AND METHOD FOR EXTRACTING, DECODING, AND 
UTILIZING HIDDEN DATA EMBEDDED IN AUDIO SIGNALS 



CROSS REFERENCE TO RELATED APPLICATIONS 
5 [1001] This application claims the benefit of U.S. Provisional Application No. 

60/200,527, filed April 27, 2000, which is incorporated herein by reference. 



BACKGROUND OF THE INVENTION 
[1002] Field of the Invention 

10 [1003] This invention generally relates to the field of encoding and decoding 

hidden data in electronic signals. More particularly, the present invention relates to a 
system and method for enabling a user to retrieve, decode, and utilize hidden data 
embedded in audio signals. 
[1004] Description of Related Art 

15 [1005] Generally, technology for embedding hidden data in electronic signals 

is well known in the art. One such technology is known as digital watermarking. 
Digital watermarking technology has developed along with the need to devise more 
effective ways for protecting copyrights of images and music made available to the 
public through the Internet. 

20 [1006] In digital watermarking, a watermark is embedded in spectral 

components of a data signal using, for example, spread spectrum techniques. This 
watermark, which is actually hidden information streams, is primarily used to protect 
against illegal use of proprietary images and/or music produced in the form of 
electronic signals. The watermark is virtually impossible to remove and is integrated 

25 throughout the signal. Digital watermarking is therefore particularly effective for 
protecting and tracking the use of electronic images and music. Although the 
watermark is not perceptible to human senses, it can be detected electronically using a 
variety of different detection and retrieval techniques. One particularly beneficial use 
of this technology is the ability to watermark audio signals. In this capacity, 

30 watermarks can be used by commercial industry, for example, to protect against the 
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illegal use of copyright protected songs or to monitor the use and broadcast of 
commercials. 

[1007] In traditional applications of watermarking, watermarks are added to a 

piece of music, or a commercial, for example, at various points in the recording 
and/or distribution cycle. In a song, a watermark might be added at the studio and 
may contain the name of the studio, date and names of the musicians, and their 
copyright statement. Another watermark may be added by the record company, 
containing their copyright statement, when a compact disk (CD) is created. Finally, 
another watermark could be added by the radio station when the CD or commercial is 
played and broadcast. Although inaudible to the listener, the virtually indestructible 
watermark remains in the audio associated audio signal. That is, the watermark 
remains embedded in the electronic version of the song or commercial. 
[1008] At any point where a user can hear the music, such as when the music 

is reproduced through an audio speaker, the listener would be able to decode the 
embedded watermark with the proper equipment. In the case of broadcast music, the 
decoded watermark could provide the user with the name of a particular song, and the 
name of the record company. The watermark could also provide the call letters of the 
radio station and any other information added to the watermark along the recording 
and/or distribution cycle of the associated audio signal. The ability to decode 
watermarks in this manner also provides a means to monitor the use, and/or amount of 
airplay of materials such as radio commercials. 

[1009] As mentioned above, techniques for embedding hidden information in 

data signals are well known in the art. For example, U.S. Patent 5,822,360, issued to 
Lee et al. on October 13, 1998 describes one such method. In Lee, auxiliary data is 
transported in an audio signal by hiding the data in the form of colored noise. 
Colored noise has a signal spectrum similar to the audio signals. Thus, the auxiliary 
data is first converted to a spread spectrum signal and the resulting audio signal is 
analyzed to determine its spectral shape. The same spectral shape is then imparted to 
the spread spectrum signal, which is combined with the audio signal for transmission. 
The spectral shaping can be performed using techniques such as linear predictive 
coding or sub-band coding techniques such as fast Fourier transforms. 
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[1010] Another technique for embedding a watermark in a signal is disclosed 

in U.S. Patent 5,905,800, issued to Moskowitz et al. on May 18, 1999. In Moskowitz, 
first a watermark key is identified. The key includes a binary sequence and 
information describing application of the binary sequence to the signal. The digital 
watermark is then encoded within the content signal at one or more locations by the 
watermarking key. 

[1011] Similarly, a number of prior art approaches are available for decoding 

an embedded watermark from an audio signal. One technique requires the audio 
signal to first be broadcast through an audio speaker, such as a radio speaker, in the 
form of sound waves. Since the watermark is inaudible to humans, a microphone is 
used to receive the associated sound waves and convert the received sound waves into 
an electrical signal. Next, the electrical signal is analyzed, using prior art signal 
processing techniques, in order to locate and decode the watermark and retrieve the 
information embedded therein. As mentioned above, the information could be the 
name of a song, the name of an associated record company, the call letters of the radio 
station, information regarding a broadcast commercial, and/or any other type of 
information. 

[1012] Given the increased dependence on wireless communications, there is 

a need to integrate the use of digital watermarking into the wireless communications 
arena. In this regard, there is also a need for a system and method that provides a user 
with the ability to detect and decode a watermark and/or other information hidden in 
an audio signal and present the decoded information to an end user. 

SUMMARY OF THE INVENTION 
[1013] Consider the business models created if a portable device, such as a 

wireless phone, was capable of decoding watermarks and presenting the resulting 
information to an end user. By way of example, a person could hear an advertisement 
or song of interest over a radio, while riding along in an automobile. In this particular 
example, the listener may desire to purchase an advertised product or may desire more 
information about the song, or the performing artist, such as where the artist will be 
performing in concert. 
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[1014] In order to obtain this information, the user could activate his mobile 

communication device, e.g. a wireless phone, and push a designated "more 
information" button on the phone. The wireless phone would be equipped with 
hardware and electronic circuitry that receives the sound heard through speakers of 
the automobile's radio. The wireless phone circuitry would process the audio signal 
and decode the watermark in order to extract certain encoded information. The 
encoded information includes contact information such as a phone number or an 
Internet uniform resource locator (URL) address associated with the song or 
commercial broadcast over the radio. That information is then displayed to the user in 
order to facilitate placement of a phone call to a designated phone number or to 
establish an Internet link using the URL. The user could then either speak to an 
operator, or interact with the web site in the usual manner, requesting concert 
information and/or ordering the product. The revenue stream associated with the 
transaction can be tracked and shared among the various entities participating in such 
an environment. 

[1015] Consistent with the principles of the present invention as embodied 

and broadly described herein, an exemplary embodiment comprises a microphone 
structured to receive sound waves representative of (i) an audio signal and (ii) hidden 
data embedded in the audio signal. The microphone converts the received sound 
waves into an electrical output signal. The system includes a processor electrically 
coupled to the microphone and configured to receive the electrical output signal in 
order to extract the hidden data and provide information represented by the hidden 
data as an output thereof. A user interface is also provided. The user interface is 
electrically coupled to the processor and configured to (i) receive a first input from 
the user and (ii) activate the processor to selectively initiate extraction of the hidden 
data in accordance with the first user input. The processor produces as an output the 
information represented by the hidden data. Finally, the system includes a user 
presentation mechanism configured to present the information represented by the 
hidden data to the user. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
[1016] The accompanying drawings, which are incorporated in and constitute 

a part of this specification, illustrate preferred embodiments of the invention and, 
together with the description, explain the objects, advantages, and principles of the 
invention. In the drawings: 

[1017] FIG. 1 is a diagram of a business model in accordance with an 

embodiment of the present invention; 

[1018] FIG. 2 is an illustration of a prior art system for providing a watermark 

in an audio signal at a content provider location; 

[1019] FIG. 3 is an illustration of a prior art system for providing a watermark 

in an audio signal at a broadcast facility; 

[1020] FIG. 4 is an illustration of an exemplary communication device and 

one audio signal source; 

[1021] FIG. 5 is an illustration of an audio source included in the 

communication device; 

[1022] FIG. 6 is a functional block diagram of an exemplary communication 

device; 

[1023] FIG. 7 is a flow diagram of the operation of an exemplary processor; 

[1024] FIG. 8 is an illustration of a processor operating in accordance with 

the present invention; 

[1025] FIG. 9 is an illustration of a display presenting decoded watermark 

information to a user; and 

[1026] FIG. 10 is an illustration of an exemplary communication device 

embedding an additional watermark in an information signal. 



Express Mail No. EK 807 713 056 US 6 PATENT 

Attorney Docket 000283 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
[1027] The following detailed description refers to the accompanying 

drawings that illustrate exemplary embodiments consistent with the present invention. 
Other embodiments are possible and modifications may be made to the embodiments 
5 without departing from the spirit and scope of this invention. Therefore, the 
following detailed description is not meant to limit the invention solely to the 
embodiments described. 

[1028] FIG. 1 illustrates a business model in accordance with an embodiment 

of the present invention. In particular, FIG. 1 shows a number of components, 
10 content providers 10, broadcast facility 20, an audio source 30, a communication 
device 40, and a base station 50. Also included is a network server 60, access to the 
Internet 70, and a public telephone network 80. Each of these components is 
discussed in greater detail below. 

[1029] The content provider 10 may, for example, be a record company or 

15 recording studio. The content provider 10 creates recording media, which includes 
audio information of interest to a system user such as songs or other material. As 
shown in greater detail in FIG. 2, the content provider 10 first creates content 
information, such as an original audio signal 11, created from a recording session of a 
musical artist. The audio signal 1 1 may be an electronic representation of a particular 

20 song, for example, the song Southern California, recorded by the musical Group 
Maze. Next, using techniques well known in the art and discussed above, a 
watermark 12 is created for embedding additional information in the audio signal 11. 
The watermark 12 may include, by way of example, identification (ID) information 
representing the name of the song, the performing artist, the name of a retail outlet 

25 where the song can be purchased, or the copyright owner's information etc. The 
watermark 12 may also include linking information, such as a phone number or URL 
in order to facilitate the user's direct purchase of the song or advertised products, or 
to facilitate some other interaction or transaction with the user. 
[1030] An embedding unit 45, that embodies the known watermarking 

30 techniques, receives the audio signal 11 and the watermark 12 as inputs, and hides or 
embeds the watermark 12 in the audio signal 11. Thus, the embedding unit 45 
produces as an output, a watermarked audio signal 13. When the embedding process 
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has been completed, the content provider 10 records the song and produces and 
output media 14, such as a CD, which contains the watermarked audio signal 13. As 
seen in FIG. 1, an embedding unit 45 may also be included in the broadcast facility 

20, and the communication device 40. 

5 [1031] The content provider 10 then supplies the output media 14 to the 

broadcast facility 20, such as a radio station having, for example, the call letters 
KGFJ. As shown in FIGs. 1 and 3, the broadcast facility 20 receives the output 
media 14 for broadcast over the airways. Further, the broadcast facility 20 may 
desire to add its own watermark 21, which could include the radio station's call 

10 letters KGFJ. Thus, the broadcast facility 20 uses its embedding unit 45 to receive 
the watermarked audio signal 13 and watermark 21, as inputs. The embedding unit 
45 produces as an output, signal 22, which includes the original watermark 12 and the 
new watermark 21. A transmission unit 23 within the broadcast facility 20, such as a 
transmitter, may then convert the signal 22 into an appropriately formatted broadcast 

15 signal 24. Once converted, the broadcast facility 20 may then radiate the broadcast 
signal 24 from the antenna 29 for broadcast over the airways. The broadcast signal 
24 could be, for example, a radio frequency (RF) or microwave signal. 
[1032] Referring again to FIG. 1, a user traveling in an automobile (not 

shown), may be listening to an audio source 30, such as a radio in the automobile. 

20 Although the audio source 30 of the present exemplary embodiment is a radio, the 
audio source alternatively could be a television 38, a public address (PA) system 39 
etc. 

[1033] Next, and as indicated more clearly in FIG. 4, the audio source 30 

receives the broadcast signal 24 through an antenna 33. When the user tunes a dial 35 

25 of the audio source 30 to an operating frequency of the broadcast facility 20, circuit 
components (not shown) of the audio source 30 demodulate the broadcast signal 24 to 
retrieve an information signal. As known in the art, the information signal is 
converted into an appropriately formatted electrical signal, and provided to speaker 
31. The speaker 31 then converts the electrical signal into sound waves 32 that the 

30 user hears in the form of the song Southern California. The sound waves 32 are thus 
representative of signal 22, shown in FIG. 3, which includes the watermarks 12 and 

21. Although the user can hear the song produced by the sound waves 32, the user 
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cannot hear the watermarks 12 and 21, which are also embedded in the sound waves 
32. 

[1034] As the user listens to the song through the speaker 31, the user may 

not be aware of the name of the song or other specifics regarding any other audio 
5 content, such as a commercial. The user may, however, desire to know the name of 
the song, the performing artist, and/or may desire to purchase a copy of the song or an 
advertised product. The user may also desire to know the call letters of the broadcast 
facility 20 that broadcast the song. In an exemplary embodiment of the present 
invention, and as illustrated in FIG. 1, the user is provided with a communication 
10 device 40 to obtain the desired information and/or order a copy of the CD or other 
material. 

[1035] Communication device 40 could be any device that has a microphone 

capable of receiving the sound waves 32 from the audio source 30, and a processor 
(e.g. processor 44 in FIG. 6) for extracting watermarks from received sound waves. 
15 Although in the exemplary embodiment of FIG. 1, the communication device 40 is a 
wireless phone, other devices, such as a lap top computer or a personal digital 
assistant (PDA), could be used. 

[1036] Referring again to FIG. 4, the communication device 40 includes a 

user interface 42 for permitting a user to either activate an information mode or enter 

20 choices regarding displayed information. For example, if the user desires additional 
information regarding a song or advertisement heard through the speaker 3 1 , the user 
can activate the user interface 42 to request additional information regarding the song 
or advertisement. The additional information would be presented to the user on a 
display 46. Additionally, instead of displaying all available information, the user 

25 could select only a representative portion for presentation on the display 46. After 
the additional information is presented to the user, the user interface 42 may also be 
operated to make specific selections pertaining to the information presented. A menu 
scheme can be implemented using the display 46 and the user interface 42 to facilitate 
selection of specific items included in the information. When the user first operates 

30 interface 42, a microphone 41 becomes active to receive the sound waves 32 and 
convert the sound waves 32 into an electrical signal. The communication device 40 
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is now set to extract the ID and linking information from the watermarks 12 and 21, 
shown in FIGs. 4 and 6-7. 

[1037] In an alternative embodiment, an audio source could be embedded in a 

communication device. In FIG. 5, for example, a communication device 40 'such as a 
cell phone, includes an audio source 30', such as a radio. In this arrangement, the 
communication device 40' could directly receive the broadcast signal 24 through an 
antenna 43b'. Here, sound waves need not be produced by a speaker 31' and then 
received by a microphone 41' prior to extraction of the ID and linking information 
from the watermarks in broadcast signal 24, as done in the embodiment of FIG. 4. 
The communication device 40' instead can be configured to demodulate the 
broadcast signal 24, properly format the demodulated signal, and provide the 
formatted signal as a direct input to a processor 44 (shown in FIG. 6). From this 
point, the communication device 40' would operate the same as communication 
device 40 in terms of extracting the ID and linking information from the watermarks 
embedded in broadcast signal 24. 

[1038] FIG. 6 is a block diagram illustrating an exemplary implementation of 

the communication device 40 shown in FIG. 4. In FIG. 6, the microphone 41 is 
electrically coupled, at least indirectly, to a processor 44. The processor 44 is 
operable to extract and decode the watermarks 12 and 21, display the associated ID & 
linking information and/or activate a network link in order to facilitate interaction 
with an entity associated with the watermarks (e.g., purchase of a CD or other 
advertised product). The microphone 41 receives the sound waves 32, and converts 
them into an electronic output signal 32a. Signal 32a is then provided to the 
processor 44. The user interface 42 of the communication device 40 is also coupled 
to the processor 44. The processor 44 provides an information output 44a to a display 
46 and can also be configured to provide a linking output 44b to an embedding unit 
45 for embedding additional watermarks. The embedding unit 45 of the 
communication device 40 provides an output signal to output mechanisms 43a and 
43b. Output mechanisms 43a and 43b, as commonly used in wireless phones, may be 
a transmitter and an antenna respectively. 

[1039] The invention, however, is not limited to use of output mechanisms 

43a and 43b, and can be practiced with alternative forms of output mechanisms. In 
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response to the output signal produced by the embedding unit 45 of the 
communication device 40, a user response signal 92 is produced and radiated from 
the antenna 43b. The user response signal 92 includes the user's response to the 
information displayed on the display 46. Specifically, this information may include 
the user's credit card number and/or other pertinent information required to interact 
with the entity associated with the watermark 12 or 21 (e.g., to purchase the CD or 
other advertised product). The user response signal 92, formatted for transmission 
across a wireless link, can also include the watermark added by the embedding unit 
45 of the communication device 40. 

[1040] FIG. 7 illustrates the internal operation of the communication device 

40. As mentioned above, when the user desires additional information regarding a 
song, advertised product, or other information heard over the speaker 31, the user 
operates the user interface 42, as shown in block 402 of FIG. 7. This process is 
shown also in FIGs. 4 and 6. When the interface 42 is operated, the processor 44 
receives the electrical signal from the microphone 41 and extracts the ID and linking 
information as indicated in block 404 of FIG. 7. 

[1041] FIG. 8 provides a more detailed view of the extraction process of the 

processor 44. As shown, the electrical signal 32a, representative of the sound waves 
32 and containing the watermarks 12 and 21 is received as an input to the processor 
44. The processor 44, using one or more techniques known in the art, removes the 
watermarks from the signal 32a. In block 406, the processor 44 then produces the 
output 44a, which includes the ID information and the linking information formatted 
for presentation on the display 46. Thus, watermark 12, containing ID information in 
the form of the name of the performing artist, and the name of the song, and linking 
information in the form of a web site, are presented as the output 44a in an exemplary 
format 12'. Similarly, watermark 21, also including ID information, is presented in 
an exemplary format 21'. Information in the formats 12' and 21' is then presented to 
the user on the display 46 as indicated in block 406 of FIG. 7, and as illustrated in 
FIG. 9. The user may now decide whether to utilize the linking information 
presented on the display 46, in order to actually establish a communications link with 
the associated entity (e.g., in order to purchase the CD or another advertised product). 
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[1042] As shown in block 408 of FIG. 7, if the user merely desires to view the 

ID information and does not wish to establish a communication link, the process ends 
in block 410. On the other hand, if the user desires to have the communication device 
40 utilize the displayed linking information to establish a communications link, the 
user may so indicate by operating the user interface 42 as indicated in block 412. An 
established communication link will permit the communication device 40 to 
automatically dial a displayed phone number, using for example, the public telephone 
network 80 shown in FIG. 1. Alternatively, the communication link may also permit 
the communication device 40 to contact an Internet web site which can be displayed, 
for example, as a URL. When the communications link is actually established, the 
communication device 40 transmits the user response signal 92, which is then 
received by an antenna 53 connected to a network resource device, such as the 
wireless communication network base station 50, shown in FIG. 1 . 
[1043] Prior to the communication link being established, the user and/or 

manufacturer of the communication device 40 may desire to add additional 
information to the processor output 44b. For example, the manufacturer may desire 
to embed a serial number of the communication device 40 so that the manufacturer 
may collect his share of the revenue as compensation for providing a communication 
device with the capability to extract embedded information. Additionally, a wireless 
service provider may also desire to collect revenues for providing a wireless network 
embodied with the features discussed above. To this end, any party in the system that 
provides a service (e.g., the artist, the CD provider, the radio station, the receiver 
manufacturer, the communication device manufacturer or service provider, and/or the 
like) may be provided with the capability to embed watermarks or other information 
for purposes such as revenue sharing. 

[1044] FIG. 10 illustrates the embedding of an additional watermark. To 

embed the additional watermark, the output 44b of the processor 44 is first provided 
to the embedding unit 45 of communication device 40, as indicated in block 414 of 
FIG. 7. In block 416, a watermark 88 is then added to the user's selection. Here, the 
embedding unit 45 of the communication device 40 receives as inputs, output 44b 
from the processor 44 and the watermark 88. The embedding unit 45 of the 
communication device 40 then provides an output response 92, which includes the 
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user's product selection and the watermark 88. As shown in block 418, the response 
signal 92 is then transmitted to the base station 50 of FIG. 1. The base station 50 
receives the user response signal 92 and extracts the serial number information from 
the signal 92. The base station 50 also establishes a communications network 
5 connection through network server 60 as indicated in block 420 and shown in FIG. 1. 
The base station 50 uses the manufacturer and/or the wireless service provider's 
information to allocate revenue according to previously negotiated agreements. 
Finally, the actual purchase of any product accomplished using the communication 
device 40 and the base station 50 may be billed to the user in a manner similar to 

10 conventional Internet purchases. 

[1045] The network server 60 facilitates communication by the 

communication device 40 with the entity identified by the URL provided in 
watermark 12. Thus, the user is able to use the communication device 40 to extract 
and decode the watermark 12 from his automobile, establish a link with the 

15 associated web site using the Internet 70 and, for example, order a CD or the 
advertised product, or otherwise interact with that web site. Correspondingly, content 
provider 10 may also transmit web page information through the Internet 70 back to 
the communication device 40 for presentation on the display 46. Alternatively, the 
network server 60 can establish a communication link between the communication 

20 device 40 and a public telephone network 80, if a telephone number was provided in 
the linking information instead of a web address, as indicated in block 422 and shown 
in FIG. 1. The user then can interact with the entity that answers that telephone at the 
opposite end of the established communication link. 

[1046] The foregoing description of the preferred embodiments provides an 

25 illustration and description, but is not intended to be exhaustive or to limit the invention 
to the precise form disclosed. Modifications and variations are possible consistent with 
the above teachings or may be acquired from practice of the invention. The foregoing 
exemplary implementations thus provide a convenient way of extracting hidden data 
from audio signals and/or interacting with an associated entity. 



